Image encoding apparatus and method

ABSTRACT

An image coding apparatus which improves the picture quality of an image. A quantization index arithmetic operation section calculates, based on motion vector residuals of macro block and the sum total of the motion vector residual outputted from a motion vector residual buffer section, a weighted average of the motion vector residual. The quantization index arithmetic operation section calculates, based on the value of activity of the macro block and the sum total of the activity outputted from an activity buffer section, a weighted average of the activity. The quantization index arithmetic operation section calculates a quantization index based on the weighted average of the vector residual or the weighted average of the activity.

TECHNICAL FIELD

This invention relates to an image coding apparatus and method, and moreparticularly to an image coding apparatus and method by which thepicture quality of an image can be improved.

BACKGROUND ART

Recently, an image compression technique represented by the MPEG (MovingPicture Experts Group) has been being developed.

FIG. 1 shows a configuration of a conventional image coding apparatus 1which uses such MPEG as mentioned above.

Referring to FIG. 1, a macro block division section 11 receives framedata as an input thereto, converts the frame data into data of a macroblock (hereinafter referred to as MB), and outputs the data of the MBobtained by the conversion to a subtraction section 19, a quantizationscale determination section 18 and a reference image production section17.

The subtraction section 19 subtracts reference image data inputted fromthe reference image production section 17 from the data of the MBinputted from the macro block division section 11 (namely, thesubtraction section 19 adds the reference image data in the reversedpolarity to the data of the MB) and outputs difference data to a DCTsection 12.

The DCT section 12 performs a DCT (Discrete Consign Transform) processfor the difference data inputted from the subtraction section 19 toconvert them into DCT coefficients and outputs the DCT coefficients to aquantization section 13.

The quantization section 13 quantizes the DCT coefficients inputted fromthe DCT section 12 with a quantization scale inputted from aquantization scale determination section 18 and outputs resulting valuesto a VLC section 14 and a decoding section 16.

The VLC section 14 performs a variable length coding process for thequantized data inputted from the quantization section 13 to convert thedata into a VLC (Variable Length Code) and outputs the VLC to atransmission buffer section 15.

The transmission buffer section 15 outputs the VLC inputted thereto tothe quantization scale determination section 18. Meanwhile, thetransmission buffer section 15 converts, when the VLCs of the entireframe are inputted thereto, the VLCs into data in a unit of a frame andoutputs the data as bit stream data to the outside.

In the decoding section 16, a dequantization section 21 dequantizes thequantized data inputted from the quantization section 13 and outputsresulting data to an inverse DCT section 22.

The inverse DCT section 22 performs an inverse DCT process for thedequantized data inputted from the dequantization section 21 intooriginal image data and outputs the original image data to an additionsection 34 of the reference image production section 17.

In the reference image production section 17, the addition section 34adds image data inputted from the inverse DCT section 22 and referenceimage data, which is inputted from a motion compensation buffer section31 and corresponding to the image data, and outputs resulting data tothe motion compensation buffer section 31 through a motion compensationsection 32.

The motion compensation buffer section 31 stores the image data inputtedfrom the addition section 34 through the motion compensation section 32as predictive image data for a next frame.

A motion detection arithmetic operation section 33 detects a motionvector corresponding to the MB inputted from the macro block divisionsection 11 and outputs the motion vector to the motion compensationsection 32.

The motion compensation section 32 performs a process for motioncompensation, based on the motion vector inputted from the motion vectorarithmetic operation section 33. In particular, the motion compensationsection 32 reads out reference image data, which is corresponding to thedata of the MB inputted to the subtraction section 19, from among thepredictive image data stored in advance in the motion compensationbuffer section 31, based on the motion vector inputted from the motiondetection arithmetic operation section 33, and outputs the referenceimage data to the subtraction section 19.

In the quantization scale determination section 18, a complexityarithmetic operation section 46 calculates, based on the VLC inputted.from the transmission buffer section 15, a complexity as a parameterrepresentative of complicacy of the VLC and outputs the complexity to acomplexity buffer section 45.

The complexity buffer section 45 temporarily stores the complexityinputted from the complexity arithmetic operation section 46 andsuitably outputs the complexity to a target code amount arithmeticoperation section 41.

The target code amount arithmetic operation section 41 calculates atarget code amount based on the complexity inputted from the complexitybuffer section 45 and outputs the target code amount to a quantizationindex arithmetic operation section 42.

The quantization index arithmetic operation section 42 calculates aquantization index based on the target code amount inputted from thetarget code amount arithmetic operation section 41 and outputs thequantization index to a quantization scale arithmetic operation section43.

An activity arithmetic operation section 44 calculates an activity as aparameter representative of complicacy of data of each of the MBsinputted from the macro block division section 11 and outputs theactivity to the quantization scale arithmetic operation section 43.

The quantization scale arithmetic operation section 43 calculates aquantization scale based on the quantization index inputted from thequantization index arithmetic operation section 42 and the activityinputted from the activity arithmetic operation section 44 and outputsthe quantization scale to the quantization section 13.

Subsequently, operation of the image coding apparatus 1 is described.

Frame data are inputted to the macro block division section 11 andconverted into data in a unit of an MB and then outputted to thesubtraction section 19.

Reference image data inputted from the reference image productionsection 17 to the subtraction section 19 are subtracted from the data ofthe MB inputted to the subtraction section 19, and data of thedifference between them is outputted to the DCT section 12.

The difference data inputted to the DCT section 12 is converted into aDCT coefficient, and the DCT coefficient is outputted to thequantization section 13. The DCT coefficient inputted to thequantization section 13 is quantized based on a quantization scaleinputted from the quantization scale arithmetic operation section 43 tothe quantization section 13 and then inputted to the VLC section 14.

The quantized data inputted to the VLC section 14 is converted into aVLC, and the VLC is outputted to the transmission buffer section 15.

The VLC inputted to the transmission buffer section 15 is outputted tothe complexity arithmetic operation section 46. When VLCs of the entireframe are inputted to the transmission buffer section 15, the VLCs areconverted into data in a unit of a frame, and the data are outputted asbit stream data to the outside.

The quantized data inputted from the quantization section 13 to thedequantization section 21 is dequantized and outputted to the inverseDCT section 22. The dequantized data inputted to the inverse DCT section22 is converted back into original image data, and the original imagedata is outputted to the addition section 34.

The image data inputted to the addition section 34 is added to thereference image data (that is, reference image data obtained by thesubtraction from the data of the MB by the subtraction section 19)stored in the motion compensation buffer section 31 in advance and thendecoded.

The decoded data of the MB is inputted to the motion compensation buffersection 31 through the motion compensation section 32 and stored aspredictive image data of an MB of a frame to be processed subsequently.

The motion detection arithmetic operation section 33 detects a motionvector corresponding to the data of the MB inputted from the macro blockdivision section 11 and outputs the motion vector to the motioncompensation section 32.

By the motion compensation section 32, a process of motion compensationcorresponding to the motion vector inputted from the motion detectionarithmetic operation section 33 is performed. In particular, from withinthe predictive image data stored in advance in the motion compensationbuffer section 31, the predictive image data at a position displaced byan amount corresponding to the motion vector from the position of the MBis read out, and the predictive image data is outputted as referenceimage data for the MB to the subtraction section 19.

The quantization scale determination section 18 performs the followingthree process, that is, so-called TM5 (Test Model 5) processes of theMPEG to calculate a quantization scale of the MB and outputs thequantization scale of the MB to the quantization section 13.

(Step 1)

The target code amount arithmetic operation section 41 calculates, basedon the value of the complexity of the VLC of the MB inputted from thecomplexity buffer section 45 and processed at the k-th (the MB processedat the k-1th is hereinafter referred to as MB-k-1), a target code amountfor the MB-k, and outputs the target code amount to the quantizationindex arithmetic operation section 42.

(Step 2)

The quantization index arithmetic operation section 42 calculates avirtual buffer occupation amount of the MB-k, based on the target codeamount for the MB-k inputted from the target code amount 41.

Further, the quantization index arithmetic operation section 42calculates a quantization index of the MB-k based on the virtual bufferoccupation amount of the MB-k and outputs the quantization index of theMB-k to the quantization scale arithmetic operation section 43.

(Step 3)

The quantization scale arithmetic operation section 43 calculates aquantization scale, based on the quantization index of the MB-k inputtedfrom the quantization index arithmetic operation section 42 and thevalue of the activity of the MB-k inputted from the activity arithmeticoperation section 44, and outputs the quantization scale to thequantization section 13.

At (step 2) of the TM5 described above, the quantization index(hereinafter referred to as Q-k) of the MB-k is calculated in accordancewith the following two expressions:d-k=d0+Bk-1−(Tk×(k-1)/mbcnt  (1)Q-k=d-k×31/r  (2)

The expression (1) is an expression for determining the virtual bufferoccupation amount (hereinafter referred to as d-k) of the MB-k.

d0 represents an initial virtual buffer occupation amount at the top ofthe frame; Bk-1 represents the generation bit amount up to the precedingMB, that is, the MB-k-1; mbcnt represents the total number of MBs perone frame; and Tk represents the target code amount for the MB-k.

The expression (2) is an expression for determining the quantizationindex Q-k of the MB-k.

r represents a parameter for adjusting the speed of response of feedbackof the quantization scale determination section 18 and is calledreaction parameter.

The quantization indices of the MBs calculated in accordance with suchexpressions (1) and (2) as given above have the following problem.

In particular, the quantization indices of the MBs calculated by thefeedback loop of the quantization scale determination section 18 arecalculated based on the prerequisite that the generation code amounts ofthe MBs are all equal to each other.

As a result, there is a problem that, where the generation code amountsof the MBs are not equal to each other, the quantization indices in theframe are not stabilized.

For example, it is assumed that a frame 51 of FIG. 2 is divided intototaling mbcnt MBs composed of an MB group 52 having a large amount ofinformation, that is, a large amount of bit generation and another MBgroup 53 having a small amount of bit generation.

Then, it is assumed that the data of the MB-1 to MB-mbcnt obtained bythe conversion are coded by the image coding apparatus 1 of FIG. 1.

In this instance, the d-1 to d-mbcnt (virtual buffer occupation amountsof the MB-1 to MB-mbcnt) calculated in accordance with the expression(1) given hereinabove are such as indicated by a curve 54 of FIG. 3.

It is to be noted that the axis of ordinate of FIG. 3 indicates thevirtual buffer occupation amount d and the axis of abscissa representsthe order k (k ranges from 1 to mbcnt) of the coding process of the MBs.

In particular, as seen from the curve 54 of FIG. 3, in the front half ofthe coding process (where k is small), the virtual buffer occupationamount d-k increases in accordance with the processing order (as kincreases). On the other hand, in the latter half of the coding process(where k is great), the virtual buffer occupation amount d-k decreasesin accordance with the processing order (as k increases toward thembcnt).

Accordingly, since the quantization index Q-k increases in proportion tothe virtual buffer occupation amount d-k as indicated by the expression(2) given hereinabove, the calculated quantization indices, Q-1 toQ-mbcnt, have values dispersed, depending upon the value of k.

In other words, the quantization indices of the MBs calculated by theimage coding apparatus 1 are not stable within the frame.

As a result, the visual perceptibility, that is, the picture quality, ofthe frame coded by the image coding apparatus 1 of FIG. 1 is degraded.

DISCLOSURE OF THE INVENTION

The present invention has been made taking such a situation as describedabove into consideration and makes it possible to improve the picturequality of an image.

An image coding apparatus of the present invention is characterized inthat the image coding apparatus comprises first arithmetic operationmeans for calculating value of a parameter representative of complicacyof block, storage means for storing the value of the parametercalculated by the first arithmetic operation means, and secondarithmetic operation means for calculating a weighted average of thevalue of the parameter stored by the storage means and calculating aquantization index of the block based on the weighted average.

In the image coding apparatus of the present invention, the parametermay be an activity representative of complicacy of the block.

In the image coding apparatus of the present invention, the parametermay be a motion vector residual of the block.

Further, the first arithmetic operation means may determine, where thevalue of the motion vector residual is higher than a predeterminedthreshold value set in advance, the threshold value as the value of themotion vector residual.

Alternatively, the first arithmetic operation means may determine, wherethe frame of the screen includes block which do not use a motion vectorand block which use a motion vector, a predetermined fixed value set inadvance as the motion vector residual of the block which does not use amotion vector.

In the image coding apparatus of the present invention, where the frameof the screen includes block which does not use a motion vector andblock which uses a motion vector, the first arithmetic operation meansmay calculate an activity of the block as the parameter of the blockwhich does not use a motion vector, and calculate a motion vectorresidual of the block as the parameter of the block which uses a motionvector.

An image coding method of the present invention is characterized in thatthe image coding method comprises a first arithmetic operation step ofcalculating value of a parameter representative of complicacy of block,a storage step of storing the value of the parameter calculated by theprocess at the first arithmetic operation step, and a second arithmeticoperation step of calculating a weighted average of the value of theparameter stored by the process at the storage step and calculating aquantization index of the block based on the weighted average.

A program of a recording medium of the present invention ischaracterized in that the program comprises a first arithmetic operationstep of calculating value of a parameter representative of complicacy ofblock, a storage step of storing the value of the parameter calculatedby the process at the first arithmetic operation step, and a secondarithmetic operation step of calculating a weighted average of the valueof the parameter stored by the process at the storage step andcalculating a quantization index of the block based on the weightedaverage.

A program of the present invention is characterized in that the programcauses a computer to execute a first arithmetic operation step ofcalculating value of a parameter representative of complicacy of block,a storage step of storing the value of the parameter calculated by theprocess at the first arithmetic operation step, and a second arithmeticoperation step of calculating a weighted average of the value of theparameter stored by the process at the storage step and calculating aquantization index of the block based on the weighted average.

In the image coding apparatus and method, recording medium and programof the present invention, a quantization index of a block is calculatedbased on a weighted average of a parameter representative of complicacyof the block.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a block diagram showing an example of a configuration of aconventional image coding apparatus;

FIG. 2 is a view showing an example of a configuration of a frame;

FIG. 3 is a view illustrating an example of a virtual buffer occupationamount of the frame of FIG. 2 calculated by the image coding apparatusof FIG. 1;

FIG. 4 is a block diagram showing an example of a configuration of animage coding apparatus to which the present invention is applied;

FIG. 5 is a flow chart illustrating a process of a quantization scaledetermination section of the image coding apparatus of FIG. 4;

FIG. 6 is a view showing an example of a configuration of a framesimilar to that of FIG. 2;

FIG. 7 is a view illustrating an example of a virtual buffer occupationamount of the frame of FIG. 6 calculated by the image coding apparatusof FIG. 4;

FIG. 8 is a view illustrating an order of a process of frames coded bythe image coding apparatus of FIG. 4; and

FIG. 9 is a block diagram showing another example of a configuration ofthe image coding apparatus to which the present invention is applied.

BEST MODE FOR CARRYING OUT THE INVENTION

FIG. 4 shows an example of a configuration of an image coding apparatus61 to which the present invention is applied. Elements which correspondto those of the conventional image coding apparatus 1 of FIG. 1 aredenoted by the same reference numerals, and description of them issuitably omitted.

As shown in FIG. 4, the image coding apparatus 61 includes, in additionto the components of the image coding apparatus 1 of FIG. 1, a motionvector residual sum total arithmetic operation section 71, a motionvector residual buffer section 72, an activity buffer section 73 and amacro block buffer section 74.

In the example of FIG. 4, when compared with the conventional example ofFIG. 1, the motion detection arithmetic operation section 33 detects amotion vector corresponding to data of each of MBs inputted from themacro block division section 11 (a motion vector of an MB) and outputsthe motion vector to the motion compensation section 32. Further, themotion detection arithmetic operation section 33 calculates a motionvector residual of each MB and outputs the motion vector residual to themotion vector residual sum total arithmetic operation section 71 and themotion vector residual buffer section 72.

The motion vector residual sum total arithmetic operation section 71calculates the sum total of the motion vector residuals of the MBsinputted from the motion detection arithmetic operation section 33 andoutputs the sum total to the motion vector residual buffer section 72.

The motion vector residual buffer section 72 temporarily stores the sumtotal of the motion vector residuals of the MBs inputted from the motionvector residual sum total arithmetic operation section 71 and the motionvector residuals of the MBs inputted from the motion detectionarithmetic operation section 33 and suitably outputs them to thequantization index arithmetic operation section 42.

The activity arithmetic operation section 44 calculates an activitycorresponding to the data of each of the MBs inputted from the macroblock division section 11 (an activity of each MB) and furthercalculates the sum total of the activities of the MBs. Then, theactivity arithmetic operation section 44 outputs the activities of theMBs and the sum total of the activities of the MBs to the activitybuffer section 73.

The activity buffer section 73 temporarily stores the activities of theMBs and the sum total of the activities of the MBs inputted from theactivity arithmetic operation section 44 and suitably outputs them tothe quantization index arithmetic operation section 42 and thequantization scale arithmetic operation section 43.

The macro block buffer section 74 temporarily stores the data of the MBsinputted from the macro block division section 11 and suitably outputsthem to the subtraction section 19.

The other configuration is similar to that in the case of FIG. 1.

Subsequently, operation of the quantization scale determination section18 of the image coding apparatus 61 of FIG. 4 is described withreference to a flow chart of FIG. 5 (since the other process is similarto that in the case of FIG. 1, description thereof is omitted).

As described hereinabove, the macro block division section 11 receivesframe data as an input thereto and converts the inputted frame data intodata of MBs.

In the present example, frame data of one frame are converted into dataof mbcnt MBs.

The macro block division section 11 outputs the data of the MB-1 toMB-mbcnt in order to the macro block buffer section 74, activityarithmetic operation section 44 and motion detection arithmeticoperation section 33.

At this time, the activity arithmetic operation section 44 calculates,based on the data of the MB-1 to MB-mbcnt inputted from the macro blockdivision section 11, activities of them (the activity of the MB-k ishereinafter referred to as activity-k) at step S101 of FIG. 5.

Further, the activity arithmetic operation section 44 calculates the sumtotal of the activity-1 to activity-mbcnt (the sum total is hereinafterreferred to as activity-sum), and outputs the activities activity-k1 toactivity-mbcnt and the sum total activity-sum of the activities to theactivity buffer section 73.

At step S102, the activity buffer section 73 stores the activitiesactivity-1 to activity-mbcnt and the sum total activity-sum of theactivities inputted from the activity arithmetic operation section 44.

On the other hand, the motion detection arithmetic operation section 33calculates, based on the data of the MB-1 to MB-mbcnt inputted from themacro block division section 11, motion vector residuals of them (themotion vector residual of the MB-k is hereinafter referred to as bdv-k)in such a manner as described hereinabove, and outputs them to themotion vector residual sum total arithmetic operation section 71 and themotion vector residual buffer section 72.

It is to be noted that, in the present example, the motion detectionarithmetic operation section 33 calculates the motion vector residualbdv-k of the MBk in the following manner.

In particular, where the motion vector residuals in the forwarddirection and the backward direction are represented by bdv-k_fwd andbdv-k_bwd, respectively, when the prediction type is the forwardprediction, the motion detection arithmetic operation section 33calculates the motion vector residual bdv-k of the MBk asbdv-k=bdv-k_fwd; when the prediction type is the backward prediction,the motion detection arithmetic operation section 33 calculates themotion vector residual bdv-k of the MBk as bdv-k=bdv-k_bwd; and when theprediction type is the bi-directional prediction, the motion detectionarithmetic operation section 33 calculates the motion vector residualbdv-k of the MBk as bdv-k=(bdv-k_fwd+bdv-k_bwd)/2.

At step S103, the motion vector residual sum total arithmetic operationsection 71 calculates the sum total (hereinafter referred to as bdv-sum)of the motion vector residuals bdv-1 to bdv-mbcnt inputted from themotion detection arithmetic operation section 33, and outputs the sumtotal bdv-sum to the motion vector residual buffer section 72.

At step S104, the motion vector residual buffer section 72 stores themotion vector residuals bdv-1 to bdv-mbcnt inputted from the motiondetection arithmetic operation section 33 and the sum total bdv-sum ofthe motion vector residuals inputted from the motion vector residual sumtotal arithmetic operation section 71.

It is to be noted that, if the processes at steps S101 to S104 come toan end prior to the processing of coding the data of the MB1, then theorder of the processes at steps S101 and S102 and at steps S103 and S104is not limited.

When the processes at steps S101 to S104 are completed, the count valueof a counter (not shown) is initialized to 1 at step S105.

Thereafter, if a quantization scale for the MB-j (hereinafter referredto as MQUANT-j) is calculated by processes at steps S106 to S109 by thequantization scale determination section 18, then the counter incrementsthe count value j only by one at step S110.

It is to be noted that details of the processes at steps S106 to S109are hereinafter described.

Then at step S111, a discrimination section (not shown) of thequantization scale determination section 18 discriminates whether or notthe count value j exceeds the mbcnt.

If it is discriminated at step S111 that the count value j exceeds thembcnt, then the quantization scale determination section 18 ends itsprocessing.

On the other hand, if it is discriminated at step S111 that the countvalue j does not exceed the mbcnt (if it is discriminated that the countvalue j is equal to or less than the mbcnt), then the quantization scaledetermination section 18 returns its processing to step S106 to repeatthe processes beginning with step S106.

For example, if the count value j is j=k, then the quantization scaledetermination section 18 calculates the quantization scale MQUANT-k bythe processes at steps S106 to S109 as described above.

In particular, at step S106, the target code amount arithmetic operationsection 41 calculates a target code amount (hereinafter referred to asTk) of the MB-k based on the complexity supplied from the complexitybuffer section 45 and outputs the target code amount to the quantizationindex arithmetic operation section 42.

At step S107, the quantization index arithmetic operation section 42calculates a virtual buffer occupation amount (hereinafter referred toas d-k) of the MB-k in the following manner based on the target codeamount Tk inputted from the target code amount arithmetic operationsection 41, and the motion vector residuals bdv-1 to bdv-k-1 and the sumtotal bdv-sum of the motion vector residuals inputted from the motionvector residual buffer section 72.

It is to be noted, however, that it is assumed that, in the presentexample, a frame is composed only of MBs which use a motion vector (suchan MB is hereinafter referred to as interMB).

First, the quantization index arithmetic operation section 42 calculatesthe sum total of the motion vector residuals of those MBs processedalready in the frame inputted from the motion vector residual buffersection 72, that is, the sum total (hereinafter referred to asbdv-sum_pk-1) from the bdv-1 to bdv-k-1.

Then, the quantization index arithmetic operation section 42 calculatesthe virtual buffer occupation amount d-k in accordance with thefollowing expression (3):d-k=d0+Bk-1−(Tk×bdvsun _(—) pk-1/bdv-sum)  (3)

It is to be noted that, similarly as in the expression (1) givenhereinabove, d0 represents an initial virtual buffer occupation amountat the top of the frame, and Bk-1 represents an amount of bit generationup to the MB-k-1.

At step S108, the quantization index arithmetic operation section 42calculates a quantization index (hereinafter referred to as Q-k) of theMB-k in accordance with the expression (2) given hereinabove and outputsthe quantization index to the quantization scale arithmetic operationsection 43.

It is to be noted that, while the quantization index arithmeticoperation section 42 receives the activities, activity-1 toactivity-k-1, and the sum total activity-sum of the activities outputtedfrom the activity buffer section 73 as inputs thereto, they arehereinafter described.

At step S109, the quantization scale arithmetic operation section 43calculates the quantization scale MQUANT-k based on the quantizationindex Q-k inputted from the quantization index arithmetic operationsection 42 and the activity activity-k inputted from the activity buffersection 73 and outputs the quantization scale MQUANT-k to thequantization section 13.

Then, the counter counts as j=k+1 at step S110 in such a manner asdescribed hereinabove, and the quantization scale determination section18 repeats the processes at steps S106 to S109 to further calculate thequantization scale MQUANT-k+1.

It is to be noted that, where the motion vector residual bdv-k (k is anarbitrary number from 1 to mbcnt) calculated by the motion detectionarithmetic operation section 33 is higher than a predetermined thresholdvalue, the motion vector residual sum total arithmetic operation section71 may calculate, in the process at step S103 described hereinabove, thesum total bdv-sum of the motion vectors using the threshold value as thevalue of the motion vector residual bdv-k and output the sum totalbdv-sum of the motion vector residuals and the threshold value as thevalue of the motion vector bdv-k to the motion vector residual buffersection 72.

As described above, the third term of the right side of the expression(1) for calculating the virtual buffer occupation amount d-k of theconventional image coding apparatus 1 of FIG. 1 is a term of dividingthe number (k-1) of those MBs which have been processed by the totalnumber (mbcnt) of the MBs in the frame.

In contrast, the third term of the right side of the expression (3)given hereinabove for calculating the virtual buffer occupation amountd-k of the MB-k of the image coding apparatus 61 of FIG. 4, to which thepresent invention is applied, is a term of multiplying the distributionof the motion vector residuals by a weighted average.

For example, it is assumed that the frame 51 of FIG. 6 (that is, a frameequivalent to the frame 51 of FIG. 2) is divided into totaling mbcnt MBscomposed of an MB group 52 having a great information amount, that is, agreat generation bit amount and another MB group 53 having a smallgeneration bit amount.

Further, it is assumed that the data of the MB-1 to MB-mbcnt obtained bythe conversion are coded by the image coding apparatus 61 of FIG. 4 towhich the present invention is applied.

In this instance, the virtual buffer occupation amounts d-1 to d-mbcntcalculated using the expression (3) given hereinabove are represented bya curve 121 of FIG. 7.

It is to be noted that the axis of ordinate and the axis of abscissa ofFIG. 7 represent the virtual buffer occupation amount d and the order kof the coding process similarly to those of FIG. 3, respectively.

As indicated by the curve 121 of FIG. 7, each of the virtual bufferoccupation amounts d-1 to d-mbcnt is calculated to a substantiallyuniform value.

In this manner, the image coding apparatus 61 of FIG. 4 can calculatethe virtual occupation amounts d-1 to d-mbcnt individually absorbing thedifference in the generation information amount (bit amount) among theMB-1 to MB-mbcnt.

Accordingly, the quantization index Q-k calculated based on the virtualbuffer occupation amount d-k (k is an arbitrary number from 1 to mbcnt)can have a stabilized value independently of the value of k.

In other words, the quantization index for each MB calculated by theimage coding apparatus 61 can be stabilized within the frame.

As a result, the image coding apparatus 61 of FIG. 4 can improve thevisual perceptibility of an image, that is, the picture quality,compared with the image coding apparatus 1 of FIG. 1.

It is to be noted that, while, in the example described above, all ofthe MBs in the frame are formed from interMBs, the frame may include anMB which does not use a motion vector (such an MB is hereinafterreferred to as intraMB).

For example, if the MBs in the frame are formed only from intraMBs, thenthe quantization index arithmetic operation section 42 of FIG. 4 mayuse, in the process at step S106 of FIG. 5 described hereinabove, notthe motion vector residuals bdv-1 to bdv-k-1 (k is an arbitrary valuefrom 1 to mbcnt) and the sum total bdv-sum of the motion vectorresiduals, but the activities activity-1 to activity-k-1 and the sumtotal activity-sum of the activities to calculate the virtual bufferoccupation amount d-k.

In other words, bdv-sum_pk-1 in the expression (3) given hereinabove maybe changed into activity-sum_pk-1 (sum total of the activitiesactivity-1 to activity-k-1) while bdv-sum is changed into theactivity-sum (sum total of the activities from activity-1 toactivity-mbcnt).

On the other hand, where the MBs in the frame include interMBs andintraMBs in a mixed state, in the process at step S103, the motionvector residual sum total arithmetic operation section 71 of FIG. 4 maycalculate the sum total bdv-sum of the motion vector residuals settingthe motion vector residual bdv-k of the intraMB-k (k is an arbitrarynumber from 1 to mbcnt) to a predetermined fixed value and output thesum total bdv-sum of the motion vector residuals and the fixed value asa value of the motion vector residual bdv-k to the motion vectorresidual buffer section 72.

It is to be noted that the fixed value is preferably set to anapproximately maximum value among the values of the motion vectorresiduals of the interMBs.

Further, in the process at step S106, the quantization index arithmeticoperation section 42 may discriminate whether the MB-k is an interMB oran intraMB and, if it is discriminated that the MB-k is an interMB, usethe motion vector residual, that is, substitute bdv-sum_pk-1 and bdv-suminto the expression (3) given hereinabove to calculate the virtualbuffer occupation amount d-k.

On the other hand, when the quantization index arithmetic operationsection 42 discriminates in the process at step S106 that the MB-k is anintraMB, it may use the activity, that is, substitute activity-sum_pk-1and activity-sum into the expression (3) given hereinabove to calculatethe virtual buffer occupation amount d-k.

In the series of processes described above, for example, where data ofeach frame are a picture of one of data of an I picture, data of a Ppicture and data of a B picture, when the data of the frames areinputted in a predetermined period (in the example of FIG. 8, 1/30 [s])into the macro block division section 11 in such an order as givenbelow, the data of the frames are coded in such an order as given below.

In the following description, an I picture is represented by I, a Ppicture by P and a B picture by B, and the order of the process of eachpicture is added to the right side.

In particular, it is assumed that data of the frames are inputted to themacro block division section 11 in the order of (B-1 (-2), B-(-1), I-0,B-1, B-2, P-3, B-4, B-5, P-6, . . . ) as shown in FIG. 8. However, thetime at which I-0 is inputted is used as a reference (0).

In this instance, at time 0, the motion detection arithmetic operationsection 33 detects the motion vector of B-(-2) and calculates a motionvector residual. Thereafter, a motion vector is detected for each oneframe in the following order, and a motion vector residual iscalculated.

In particular, with reference to time 0, a motion vector is detected inthe order of (B-(-2), B-(-1), I-0, P-3, B-1, B-2, P-6, B-4, B-5, . . . )for each one frame, and a motion vector residual is calculated.

In this manner, prior to a B picture (for example, B-1 and B-2, B-4 andB-5, and so forth), a motion vector of a P picture (for example, P-3with respect to B-1 and B-2, or P-3 with respect to B-4 and B-5) isdetected and a motion vector residual is calculated.

Then, in the order wherein the motion vectors described above aredetected and the motion vector residuals are calculated, that is, in theorder of (B-(-2), B-(-1), I-0, P-3, B-1, B-2, P-6, B-4, B-5, . . . ),data of the frames are coded and outputted from the transmission buffersection 15 (in order words, before coding is performed, arithmeticoperation of motion vectors and residuals of them necessary for suchcoding must be completed).

It is to be noted that the timing at which data of the frame of B-(-2)are coded and outputted from the transmission buffer section 15 is thetime 1, and thereafter, data of the frames are processed and coded inthe order described above for each frame.

While the series of processes described above can be executed byhardware, it may otherwise be executed by software. In this instance,for example, an image coding apparatus 131 is formed from such apersonal computer as shown in FIG. 9.

Referring to FIG. 9, a CPU (Central Processing Unit) 141 executesvarious processes in accordance with a program stored in a ROM (ReadOnly Memory) 142 or a program having been loaded from a storage section148 into a RAM (Random Access Memory) 143. In the RAM 143, also datanecessary for the CPU 141 to execute various processes and so forth arestored suitably.

The CPU 141, ROM 142 and RAM 143 are connected to each other by a bus144. Also an input/output interface 145 is connected to the bus 144.

An inputting section 146 comprising a keyboard, a mouse and so forth, anoutputting section 147 comprising a display unit such as a CRT or an LCDunit and a speaker or the like, a storage section 148 formed from a harddisk or the like, and a communication section 149 formed from a modem, aterminal adapter or the like are connected to the input/output interface145. The communication section 149 performs a communication processthrough a network including the Internet.

Also a drive 150 is connected to the input/output interface 145 asoccasion demands, and a magnetic disk 161, an optical disk 162, amagneto-optical disk 163, a semiconductor memory 164 or the like issuitably loaded into the drive 150 and a computer program read out fromthose is installed into the storage section 148 as occasion demands.

Where the series of processes is executed by software, a program whichconstructs the software is installed from a network or a recordingmedium into a computer incorporated in hardware for exclusive use or,for example, a personal computer for universal use which can executevarious functions by installing various programs.

The recording medium is formed as a package medium such as, as shown inFIG. 9, a magnetic disk 161 (including a floppy disk), an optical disk162 (including a CD-ROM (Compact Disc-Read Only Memory) and a DVD(Digital Versatile Disk)), or a magneto-optical disk 163 (including anMD (Mini-Disk)), or a semiconductor memory 164 which has the programrecorded thereon or therein and is distributed in order to provide theprogram to a user separately from an apparatus body. Else, the recordingmedium is formed as a ROM 142 having the program stored therein, a harddisk included in the storage section 148, or the like which are providedto a user in a state wherein they are incorporated in the apparatus bodyin advance.

It is to be noted that, in the present specification, the steps whichdescribe the program stored in a recording medium may be but need notnecessarily be processed in a time series in the order as described, andinclude processes which are executed in parallel or individually withoutbeing processed in a time series.

INDUSTRIAL APPLICABILITY

As described above, according to the image coding apparatus and methodof the present invention, since a quantization index of a block iscalculated based on a weighted average of a parameter representative ofcomplicacy of the block, the picture quality of an image can beimproved.

1. An image coding apparatus for coding an image in a unit of a blockcomprises: first arithmetic operation means for calculating value of aparameter representative of complicacy of the block, wherein the firstarithmetic operation means determines, when the value of the motionvector residual is higher than a predetermined threshold value set inadvance, the threshold value as the value of the motion vector residual;storage means for storing the value of the parameter calculated by saidfirst arithmetic operation means; and second arithmetic operation meansfor calculating a weighted average of the value of the parameter storedby said storage means and calculating a quantization index of the blockbased on the weighted average.
 2. An image coding apparatus according toclaim 1, wherein the parameter is an activity representative ofcomplicacy of the block.
 3. An image coding apparatus according to claim1, wherein the parameter is a motion vector residual of the block.
 4. Animage coding apparatus according to claim 3, wherein said firstarithmetic operation means determines, where the frame of the screenincludes block which does not use a motion vector and block which uses amotion vector, a predetermined fixed value set in advance as the motionvector residual of the block which does not use a motion vector.
 5. Animage coding apparatus according to claim 1, wherein where the frame ofthe screen includes block which does not use a motion vector and blockwhich uses a motion vector, said first arithmetic operation meanscalculates an activity of the block as the parameter of the block whichdoes not use a motion vector, and calculates a motion vector residual ofthe block as the parameter of the block which uses a motion vector. 6.An image coding system comprising: at least one processor; and at leastone memory, coupled to the at least one processor, the at least onememory storing a method for an image coding apparatus which codes animage in a unit of a block, the method comprises: a first arithmeticoperation step of calculating value of a parameter representative ofcomplicacy of the block, wherein the first arithmetic operation stepdetermines, when the value of the motion vector residual is higher thana predetermined threshold value set in advance, the threshold value asthe value of the motion vector residual; a storage step of storing thevalue of the parameter calculated by the process at the first arithmeticoperation step; and a second arithmetic operation step of calculating aweighted average of the value of the parameter stored by the process atthe storage step and calculating a quantization index of the block basedon the weighted average.